For practical applications in retail management and surveillance, overhead wide-angle cameras are used to view the entire scene and to extract global information. The cameras can be used to track people and their traffic patterns. Applications include improving store design and optimizing advertising effectiveness by cataloguing customer behavior in retail stores, providing face cataloguing for surveillance, and providing input for an enhanced reality system. Each of the foregoing applications can benefit from knowledge of the person's activities; and, one of the obvious first steps is to understand the pose of the person. “Pose” is used herein to refer to the direction or orientation that a rigid body is facing.
Overhead wide-angle cameras introduce several challenges to the problem of pose estimation. These challenges include lens distortion artifacts and low-resoluion imagery, particularly of the head. In addition, images from overhead wide-angle cameras introduce a ‘virtual’ or relative pose that differs from the true pose (i.e., orientation). FIG. 1 shows a composite camera image composed from several images taken by one camera of an individual who is looking in one absolute direction, specifically towards the front wall. The composite image was created by superimposing 4 images, taken from the same camera with fixed orientation, of a person in 4 different locations in the room. The individual is looking in the same absolute direction in all of the images; however, the individual ‘appears’ to be looking in different directions due to the differences in location with respect to the camera. Specifically, in the different images included in the composite image, the individual appears to be looking down or to the side due to the relative figure orientation with respect to the wide-angle overhead camera. Such apparent differences in pose can lead to inaccurate information being provided for the intended cataloging of customer behavior.
What is needed, and what is therefore an object of the present invention, is a system and method to provide estimation of the absolute pose of a rigid body as seen from different locations with respect to one or multiple cameras, including wide-angle overhead cameras.
It is another object of the invention to provide a system and method to estimate absolute pose of a person's head as the person moves around in a three dimensional space, or when the person is located at different positions, as seen from one or multiple cameras, including wide-angle overhead cameras.
Another object of the invention is to provide a system and method to estimate the absolute pose/orientation and the position in 3D space, thereby estimating the anchored viewing ray, (i.e., for a person, which way are they looking and what is it possible for them to see in the space).
Yet another object of the invention is to provide absolute pose estimation by integration of low-resolution imagery from one or multiple cameras.